Learning meanings for sentences

Consider an English sentence of length n, say “Cats like to chase mice” with n = 5. Suppose that we are given a binary tree representing the syntactic structure of the sentence. Each word is a leaf node of the tree, and there are n− 1 internal nodes. Each internal node covers a phrase of two or more consecutive words. We will associate a column vector in R with each node, to represent the meaning of the corresponding phrase. A typical value for the dimensionality d is 100. The meaning of each word is initialized to be a random vector in R. This means that we create a fixed lexicon containing a random vector for each word. Each random vector is generated independently from a Gaussian of dimension d with mean zero and diagonal covariance matrix σI . Each time the same word is used in any sentence, the same vector is used as its meaning. The meaning of a phrase is a function of the meanings of its two components. This is called a compositional approach to semantics. Let the node k have children i and j, whose meanings are xi and xj . The meaning of node k is